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DEASEL : An Expert System for Software Engineering 


by Jon D. Valett and Andrew Raskin 


ABSTRACT 

For the past ten years* the Software Engineering Laboratory Cl] 
(SEL) has been collecting data on software projects carried out 
in the Systems Development Branch of the Flight Dynamics Division 
at NASA’s Goddard Space Flight Center. Through a series of 
studies using this data* much knowledge has been gained on how 
software is developed within this environment. Two years ago 
work began on a software tool which would make this knowledge 
readily available to software managers. Ideally* the Dynamic 
Management Information Tool (DynaMITe) will aid managers in 
comparison across projects* prediction of a project’s future* and 
assessment of a project’s current state. This paper describes an 
effort to create the assessment portion of DynaMITe. 


1.0 Backround 

Assessing the state of a software project during development 
is a difficult problem* but its solution contributes to the 
success of the project. By determining a project’s weaknesses 
early in its life cycle, problems can be dealt with quickly and 
effectively. For the software manager to perform this assessment 
he needs easy access to detailed* accurate information 
(knowledge) regarding past projects within the development 
environment. He then incorporates this Information with his own 
knowledge of software engineering to make some assessment of a 
project’s strengthes and weaknesses. The DynaMITe Expert Advisor 
for the SEL (DEASEL) Is the first version of an expert system 
that attempts to simulate this process. 

2.0 Developing and Using Rules 

Basically* DEASEL assesses an ongoing project by attempting 
to answer a simple question such as "How is my project doing?!' 

To answer this question DEASEL utilizes a knowledge base of rules 
for evaluating software projects. This knowledge base consists 
of rules derived from two sources: the SEL database and 

experienced software managers. DEASEL uses these rules along 
with data on the project of interest, to give the manager a 
relative rating of the quality of that project. 
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2.1 Corporate Memory 

Of course; a major effort in the development of the DEASEL 
system was the actual collection of knowledge. To derive rules 
from the corporate memory* former studies [2,3,4,5,6,7,81 
performed by the SEL were reviewed to find relationships that 
affect the quality of a software project. That is, many studies 
of data concerning the SEL environment have been done within the 
last ten years. These studies give some idea of the cause and 
effect of technologies and methodologies on a software project. 
Thus, relationships like ’'increasing tool use will Increase 
productivity" are found. Because of the interdependencies amoung 
the items the strength of each relationship is then determined. 
For example, many different factors may influence productivity, 
therefore the determination of which of these have the most and 
which the least influence must be made. This has been a long and 
difficult process because of the amount of data and the problems 
with determining what data is relevant to the assessment process. 

2.2 Knowledge from Software Managers 

The other source of knowledge is the experienced software 
managers, who have certain "rules of thumb" they use to evaluate 
a software project. They are questioned to obtain this 
subjective information which is then used along with the more 
objectl ve -material to produce the knowledge base. Again the 
determination of the st rengthes of the relationships must be 
performed. The entire process of collecting knowledge is long 
and difficult and has only just begun for the DEASEL project. 


2.3 Representing the Rules 

After collecting a preliminary set of knowledge, thought 
began on how to actually represent this knowledge. The initial 
work on knowledge representation for DEASEL was directed at using 
standard expert system techniques. Including if-then production 
rules. But soon the discovery was made that knowledge regarding 
the assessment of a software project's development is more 
naturally represented in a different manner. In fact, the 
overall conclusion drawn from an assessment is quite different 
from that drawn by a traditional expert system. The difference 
lies in the type of question answered by DEASEL. The traditional 
medical expert system, such as the often cited MYCIN [9], 
answers a question like "What disease does patient X have?" 

Then, given some data on the patient the system determines the 
disease. DEASEL, on the other hand, must answer the question 
"How is project X doing?" Thus, It must give a rating to the 
system based on the facts given to it. The analagous question in 
the medical domain would be "How Is patient X's health?" 

In order for DEASEL to answer the question "How is project X 
doing?", it needs two different types of knowledge. The first 
type of knowledge is the assertions which relate to the specific 
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project in question. This includes the facts known about the 
project as it currently stands. The second type of knowledge is 
the detailed representation of how different facts affect the 
overall development process of a project. These are the more 
general "rules” on what affects the quality of a software 
project. These rules are set up based on the knowledge described 
earl ier from the data base and the software manager. They are 
used to describe all of the factors which affect a software 
project's quality and all the sub-factors that affect those 
factors* etc. For this reason this system of knowledge 
representation, which is unique to DEASEL, is called factor- 
based. Each rule in the factor-based representation scheme 
specifies a system and its factors (sub-systems) and the weight 
(strength of the relationship) each factor has on the system. 
Thus, between the specific assertions about the project and the 
general rules concerning software development within the SEL 
environment DEASEL can rate a project. 


2.4 An Exampl e Rul e 

To explain how this rating process works, here is an example 
rule from DEASEL's knowledge base: 

The factors that affect Computer_En v i ronment_Stab i 1 ity are 


1) Operating__System_Stabil ity .3 

2) Software_Tool_Stabiltiy .2 

3) Hardware_Stab i 1 i ty .4 

4) Computer_Env_Proc_Stab i 1 i ty .1 


The number associated with each factor is a weight, and the sum 
of the weights must always total one. This rule states that the 
four listed factors have an affect on the quality of the 
Computer_En v i ronment_Stab i 1 i ty. The rule's weights indicate that 
Hardware_Stab i 1 1 ty is the most Important factor in the assessment 
of Computer_Env i ronment_Stab i 1 ity, while 

Computer_En v_Proc_Stab 11 ity is the least important factor. 

DEASEL uses the ratings of all four factors to determine a rating 
for Computer_Env i ronment_St ab i 1 ity. 


2.5 Deriving Conclusions 

DEASEL's overall assessment process consists of trying to 
assign a rating to each of the quality indicators specified via 
the knowledge base. Obviously just answering the question "How 
is project X doing?" will not give the manager specific enough 
information about his project. Therefore, the knowledge base 
specifies the top level factors DEASEL should rate. Currently, 
the knowledge base has four such quality indicators: 
reliability, predictability, stability, and controlled 
development. Thus DEASEL actually gives information (a rating) 
on each of these four Indicators which gives the manager an 
assessment of how his project is doing in these areas. In order 
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to rate these four factors DEASEL must find the rules which 
relate to these factors and assign a rating to these rules. That 
Is, DEASEL reaches a conclusion on what It believes Is the rating 
of these Indicators. For DEASEL to do this It must first reach 
the conclusions on the factors which affect these indicators. Of 
course, these factors may have rules which specify their 
assessment, so this process continues until all of the necessary 
conclusions are reached. 

DEASEL reaches conclusions In one of three ways; 

1) The conclusion can be an assertion from the knowledge 
base. 

2) DEASEL can infer the conclusion based on other 
conclusions and Its rule base. 

3) If both 1) and 2) fall, it can ask the user to supply 
the conclusion. 

The three types of conclusions combine to allow DEASEL to make 
Its assessment of the supplied quality Indicators. The basic 
process is to first find a ru le for one of the quality Indicators 
then to resolve all of the conclusions necessary to reach a 
conclusion for that Indicator. This process continues by 
reaching conclusions in each of the three ways, until all the 
conclusions are resolved. 

To fully understand the rating process one must also 
understand how these conclusions are reached. A conclusion Is 
reached when a rating has been assigned to a factor In the 
knowledge base. A rating Is defined as a number between zero and 
one, the higher the rating the better the factor’s qual Ity. A 
rating of .5 would be average or normal. Note that the ratings 
always Indicate quality, for example a rating of .7 for error 
rate as a factor would Indicate a lower than normal error rate. 

In addition, every conclusion has an associated certainty. A 
certainty is the probability that the conclusion's rating is 
correct within some fixed delta. Currently, DEASEL sets delta at 
0 . 1 . 

All three types of conclusions have both a rating and a 
certainty. Type 1 conclusions are really the assertions 
described earlier. Currently, the asssert i ons are entered by 
hand into the knowledge base. In the future this process w 1 1 1 be 
automated and wll 1 be done by the DynaMITe tool, via the SEL data 
base. The certainties for these conclusions are generally very 
high (around .9) because the ratings are basically comparisons 
between real data and average or normal numbers. Conclusions of 
type 2 are computed using the following formulae; 

Rating = (Rating of f actor ( 1 ) x Weight of factor(O) 

* 

Certainty = V (Certainty f actor ( 1 ) x Weight of factor(l)) 

i-i 

where n is the number of factors In the rule 

Thus, a rule for a certain factor is given a conclusion by using 
these formulae to calculate its rating and certainty. The schema 
used here should look familiar to anyone with knowledge of 
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probability. In Its typical application, however, each of the 
factors In the system being rated must be Independent* In the 
complex and unfamiliar domain of software engineering, such an 
assumption may be Incorrect. Our computations could therefore be 
slightly or grossly In error depending on how much the knowledge 
base violates this constraint. Future DEASEL knowledge engineers 
must keep this In mind when creating and modifying the rule base. 
Type 3 conclusions are necesssary when the system cannot use type 
1 or type 2 conclusions. In order for the system to complete an 
assessment It must have conclusions for all the factors In the 
knowledge base. Since expert systems must deal with Incomplete 
knowledge, whenever DEASEL cannot reach a conclusion for a factor 
It assumes a normal rating (.5) with a certainty of .2. Note 
that the .2 is the probabll Ity that the rati ng will be correct 
within + or - delta* which in effect makes for a meaningless 
conclusion. Whenever DEASEL Is forced to do this. It makes a 
note to ask the user If the conclusion can be provided. Thus, 
the user can later provide the answers to questions about the 
Incomplete knowledge. Once these questions are answered, DEASEL 
gives the rating supplied by the user a certainty of 1.0. 


2.6 Current DEASEL Capabilities 


The capabilities of the current DEASEL system Include 
allowing the user to obtain an assessment of his project, if some 
assertions exist for that project. After the Initial assessment 
is given the user has three options 1) asking for an 
exp 1 anant Ion, 2) answering questions about his project, and 3) 
playing what-if games. For any conclusion, the user can ask for 
an expalnantion of how the conclusion was reached. The 
explanation consists of the conclusions DEASEL reached about the 
factors of the original conclusion. That Is, the user Is able to 
ask DEASEL what caused It to reach any specific rating for any 
factor. This process can continue as the user asks for 
explanations of the factors previously reported on, and so on. 
Earl ler we mentioned that DEASEL makes a note of type 3 
conclusions. The user may opt to answer these questions as he 
wishes. He may also respond to the questions by indicating he 
does not know the answer. In this case, DEASEL maintains the 
meaningless conclusion reached earlier. Answering questions is 
encouraged because it leads to more certain conclusions. What-if 
games aid the manager In evaluating the effects of changes he may 
wish to make In his project. This process allows the user to 
enter controls Into the system, by actually changing conclusions. 
That Is, the user can see what will happen if he changes certain 
conclusions In the knowledge base. After changing one or more 
conclusions he can then reassess the project, to determine the 
affects of these changes. This is an Important feature of the 
DEASEL system, because it allows the manager to determine how he 
might be able to Improve his software project. 
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3.0 Summary 

Although the current version of DEASEL does begin to attack 
the problem of project assessment# much more work is needed to 
make the system a useful tool. Three potential directions exist 
for future work: adding to and verifying the rule base# 

verifying the accuracy of the assessment process# and automating 
the creation of the assertion portion of the rule base. A1 1 of 
these areas will require time and effort to complete# but are 
necessary for successfully determining the validity of this 
project. Obviously# DEASEL is but an initial attempt at solving 
the problem of automating the process of assessing the state of 
an ongoing software project. DEASEL has# however# given some 
insight into the problem and ways to solve it. Hopefully this 
initial work will lead to techniques for solving the problem more 
complete! y . 
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COLLECTING KNOWLEDGE 

From Corvorate Memory I From Software Manager 
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TO ANSWER THE QUESTION . 
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• Controlled Development 



THE KNOWLEDGE BASE 
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THE RATING PROCESS 

A Simple Example 




r-- 

» 


CD 


-o 

rr> 

O 

o 

• 

'o 

* MM 

a> 

s 

"o 

a 

cm 

CO 

CD 

CD 

£= 

CD 

O 

*CD 

_cr 

a? 

o 

Q 


J. Valett 
NASA/GSFC 
16 of 21 



THE RATING PROCESS 
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THE RATING PROCESS 

A Simple Exomple 
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